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CLAIMS 



1 1. A method for assessing adequacy of message flow 

2 testing, comprising: 

3 defining coverage criteria for testing a message 

4 flow through a set of message flow elements; 

5 determining a message-flow-coverage-goal for the 

6 message flow with respect to the coverage criteria; 

7 designing a test suite responsive to the message- 

8 flow-coverage-goal; 

9 applying the test suite to the message flow to 

10 generate a coverage result for the set of message flow 

11 elements; and 

"q 12 comparing the coverage result with the message-f low- 

0 13 coverage-goal. 

H . 1 2. A method according to claim 1, wherein the 

7\ 2 message flow comprises a message-oriented software 

w 3 program wherein a message is a primary architectural 

p 4 element. 

W 

pi 3. A method according to claim 1, wherein the 

^ 2 message flow is comprised in a message-oriented 

jffj 3 middleware application. 

1 4. A method according to claim 1, wherein the 

2 message flow comprises a visual program describing 

3 processing logic as a directed graph and wherein the 

4 message flow comprises an independent function, outside 

5 of a message sender or message receiver. 

1 5. A method according to claim 1, wherein the 

2 message flow comprises a program processing a message 

3 produced by an application chosen from one of a group of 

4 applications comprising a client application and a server 

5 application. 

1 6. A method according to claim 5, wherein the 
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2 application comprises a raessage-sending-application and a 

3 message-receiving-application executing on different 

4 hardware platforms. 

1 7. A method according to claim 6, wherein the 

2 message-sending-application and the message-receiving- 

3 application are implemented using different software and 

4 architectural paradigms. 

1 8. A method according to claim 1, wherein the 

2 message flow elements comprise at least one element 

3 chosen from messages, nodes, connections, terminals, 

4 statements, conditions, external resources, and 

5 exceptions. 

p "1 9. A method according to claim 1, wherein defining 

O 2 the coverage criteria comprises selecting a coverage 

1^ 3 model from at least one of processing node coverage, 

4* 4 terminal coverage, connection coverage, path coverage, N- 

Cp 5 node coverage, statement coverage, multiple condition 

q 6 coverage, exception coverage, external resources 

W 7 coverage, and message content coverage. 

o 

Hi 10. A method according to claim 1, wherein 

jfj 2 determining the message-flow-coverage-goal comprises 

3 establishing a required coverage level for at least one 

4 of the coverage criteria. 

1 11. A method according to claim 1, wherein applying 

2 the test suite to the message flow comprises: 

3 performing an execution of at least a part of the 

4 test suite; and 

5 evaluating an attained coverage level resulting from 

6 the execution. 

1 12. A method according to claim 11, wherein 

2 evaluating the attained coverage level comprises: 

3 visiting one or more nodes during the execution of 
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the at least part of the test suite; 

comparing a number of processing nodes visited to a 
total number of all processing nodes in the message flow; 
and 

computing a processing node coverage metric 
responsive to the comparison. 

13. A method according to claim 12, wherein 
visiting the one or more nodes comprises entering a 
simple node. 

14. A method according to claim 12, wherein 
visiting the one or more nodes comprises entering a 
compound node and visiting zero or more nodes in a 
constituent sub-flow. 

15. A method according to claim 11, wherein 
evaluating the attained coverage level comprises: 

traversing one or more terminals during the 
execution of the at least part of the test suite; 

comparing a number of processing node terminals 
traversed to a total number of processing node terminals 
in the message flow; and 

computing a terminal coverage metric responsive to 
the comparison. 

16. A method according to claim 11, wherein 
evaluating the attained coverage level comprises: 

traversing one or more connections during the 
execution of the at least part of the test suite; 

comparing a number of connections traversed to a 
total number of connections in the message flow; and 

computing a connection coverage metric responsive to 
the comparison. 

17. A method according to claim 11, wherein 
evaluating the attained coverage level comprises: 
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3 selecting a group of one or more sets of N-nodes 

4 from the message flow, wherein N comprises any whole 

5 number less than or equal to a total number of nodes in 

6 the message flow; 

7 performing at least one execution of the message 

8 flow so as to determine a number of nodes visited in each 

9 of the one or more sets during the at least one 

10 execution; 

11 generating a respective set coverage result for each 

12 of the one or more sets, responsive to the number of 

13 nodes visited; 

14 determining a number of covered-sets, responsive to 
y. 15 the set coverage results; 

5:16 comparing the number of covered-sets to a total 

j0 17 number of sets in the group; and 

Jgl8 computing an N-node coverage metric responsive to 

"J! 19 the comparison. 

^ 1 18. A method according to claim 11, wherein 

W 2 evaluating the attained coverage level comprises: 

O 

^ 3 performing zero or more runtime exceptions during 

y 4 the execution of the at least part of the test suite; 

fy 

5 comparing a number of runtime exceptions performed 

6 to a total number of all runtime exceptions in the 

7 message flow; and 

8 computing an exception coverage metric responsive to 

9 the comparison. 

1 19. A method according to claim 11, wherein 

2 evaluating the attained coverage level comprises: 

3 visiting zero or more failure terminals during the 

4 execution of the at least part of the test suite; 

5 comparing a number of failure terminals visited to a 

6 total number of all failure terminals in the message 

7 flow; and 
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8 computing an exception coverage metric responsive to 

9 the comparison. 

1 20. A method according to claim 11, wherein 

2 evaluating the attained coverage level comprises; 

3 executing zero or more node statements at least once 

4 during the execution of the at least part of the test 

5 suite; 

6 comparing a number of node statements executed to a 

7 total number of node statements in the message flow; and 

8 computing a statement coverage metric responsive to 

9 the comparison. 

;j _ 1 21. A method according to claim 20, wherein 

O 2 computing the statement coverage metric comprises 

D 

,q 3 comparing a number of node statements executed in a 

*Z 4 single node to a total number of node statements in the 

M= 5 single node. 

en 

si 22. A method according to claim 20, wherein 

O 

Hi 2 computing the statement coverage metric comprises 

H 3 comparing a number of node statements executed in a 

p 4 compound node's constituent sub-flows to a total number 

- 

!W 5 of node statements in the compound node's constituent 

6 sub-flows. 

1 23. A method according to claim 11, wherein 

2 evaluating the attained coverage level comprises: 

3 . assessing an achievement of true and false values 

4 for each of zero or more boolean sub-expressions 

5 independently during the execution of the at least part 

6 of the test suite; 

7 comparing the achievement for the one or more 

8 boolean sub-expressions to a total number of boolean sub- 

9 expression values possible in the message flow; and 

10 computing a multiple condition coverage metric 

11 responsive to the comparison. 
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24. A method according to claim 23, wherein 
computing the multiple condition coverage metric 
comprises comparing a number of achieved true and false 
values for zero or more boolean sub-expressions in a node 
to a total number of boolean sub-expression values 
possible in the node. 

25. A method according to claim 23, wherein 
computing the multiple condition coverage metric 
comprises comparing a number of achieved true and false 
values for zero or more boolean sub-expressions in a 
compound node's constituent sub-flows to a total number 
of boolean sub-expression values possible in the compound 
node's constituent sub-flows. 

26. A method according to claim 11, wherein 
evaluating the attained coverage level comprises: 

assessing a number of values assumed by each of one 
or more fields in a message during the execution of the 
at least part of the test suite; 

comparing the number of values assumed to a total 
number of possible values for each field in the message; 
and 

computing a message content coverage metric 
responsive to the comparison. 

27. A method according to claim 26, and comprising 
computing a strong message content coverage metric 
wherein the total number of values possible for each 
field in the message comprises a cross product of the 
message fields with their possible values. 

28. A method according to claim 26, and comprising 
computing a weak message content coverage metric wherein 
the total number of values possible for each field in the 
message comprises a sum of the number of possible values 
for each message field independently. 
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1 29. A method according to claim 11, wherein 

2 evaluating the attained coverage level comprises: 

3 traversing one or more paths during the execution of 

4 the at least part of the test suite; 

5 comparing a number of paths traversed to a total 

6 number of paths in the message flow; and 

7 computing a path coverage metric responsive to the 

8 comparison. 

1 30. A method according to claim 11, wherein 

2 evaluating the attained coverage level comprises: 

3 performing zero or more types of accesses to an 

4 external resource during the execution of the at least 

5 part of the test suite; 

p 6 comparing a number of types of accesses performed to 

ff 7 a total number of types of accesses in the message flow; 

<$> 8 and 

'bJk 

0=1 9 computing a external resources coverage metric 

JL.,10 responsive to the comparison. 

Si 31. A method according to claim 1, wherein 

U 

H 2 designing the test suite responsive to the message-f low- 

O 

f|j 3 coverage-goal comprises: 

4 identifying an initial test suite for the message 

5 flow; 

6 assessing a coverage level achieved by the initial 

7 test suite; and 

8 adding additional tests to the initial test suite so 

9 as to increase the coverage level. 

1 32. A method according to claim 1, wherein applying 

2 the test suite comprises generating message-f low- 

3 coverage-reports . 

1 33. A method according to claim 29, and comprising 

2 integrating the message- flow-coverage-reports into a 



IL9-2001-0009 



46 



41485S6 



IBM CONFIDENTIAL 



3 visual message flow development environment. 

1 34. A method according to claim 1, and comprising 

2 reporting coverage graphically using at least one 

3 graphical element chosen from a set of color, shading, 

4 highlighting, graphing, fonts, line styles, icons, and 

5 labels. 

1 35. A method according to claim 1, and comprising 

2 reporting coverage via at least one medium chosen from 

3 hard-copy media and electronic media. 

1 36. A method according to claim 1, wherein 

2 generating the coverage result comprises collecting 

3 coverage data using at least one method of data 

O 4 collection chosen from message flow instrumentation and 

O 

:.g 5 data logging. 

c 

jp 1 37. Apparatus for assessing adequacy of message 

s m 2 flow testing, comprising a computer system which is 

*_ 3 adapted to define coverage criteria for testing a message 

y 4 flow through a set of message flow elements, determine a 

5 message-flow-coverage-goal for the message flow with 



Cj 6 respect to the coverage criteria, design a test suite 

ru 

7 responsive to the message-flow-coverage-goal, apply the 

8 test suite to the message flow to generate a coverage 

9 result for the set of message flow elements, and compare 
10 the coverage result with the message-flow-coverage-goal. 

1 38. A computer software product for assessing 

2 adequacy of message flow testing, comprising a computer- 

3 readable medium having computer program instructions 

4 recorded therein, which instructions, when read by a 

5 computer, cause the computer to define coverage criteria 

6 for testing a message flow through a set of message flow 

7 elements, determine a message-flow-coverage-goal for the 

8 message flow with respect to the coverage criteria, 
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design a test suite responsive to the message-flow- 
coverage-goal, apply the test suite to the message flow 
to generate a coverage result for the set of message flow 
elements, and compare the coverage result with the 
message-flow-coverage-goal . 



IL9-2001-0009 



48 



